Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PaddingFree #113

Closed
wants to merge 7 commits into from
Closed

PaddingFree #113

wants to merge 7 commits into from

Conversation

RhuiDih
Copy link

@RhuiDih RhuiDih commented Jun 28, 2024

No description provided.

@RhuiDih RhuiDih force-pushed the dev/model_mod branch 3 times, most recently from dc9d768 to 2868eab Compare June 28, 2024 08:40
@fabianlim
Copy link
Contributor

fabianlim commented Jul 1, 2024 •
edited
Loading

Performance testing:

Legend:

  • ilab-granite: using dolomite model in ilab training script
  • ilab-llama: using huggingface model in illab training script
  • hf-llama: using the padding free introduced in this PR

image

After removing dropout from Dolomite, we can achive the same MT bench score

image

fabianlim and others added 2 commits July 2, 2024 08:08
@fabianlim
disable padding in multipack
1c89a8b 
Signed-off-by: Yu Chin Fabian Lim <flim@sg.ibm.com>
@RhuiDih
Merge pull request #1 from fabianlim/dev/model_mod
2d58cbf 
disable padding in multipack
@fabianlim
Copy link
Contributor

This may be replaced by the official HF solution once its merged huggingface/transformers#31446

@Maxusmusti Maxusmusti closed this Sep 27, 2024
@mergify mergify bot added the ci-failure label Sep 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
ci-failure
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@RhuiDih @fabianlim @Maxusmusti